Image generation device, image display system, and information presentation method

ABSTRACT

Disclosed herein is an image generation device including: a space construction section that constructs a virtual world to be displayed on a head-mounted display; an image generation section that generates, based on a position and a posture of a head of a player wearing the head-mounted display, a display image representing the virtual world in a field of view corresponding to a point of view of the player, and causes the head-mounted display to display the generated display image; and a visitor information acquisition section that acquires information regarding presence of a visitor without a head-mounted display in a space where the player is able to move. While the visitor is present, the space construction section displays an object indicating the presence of the visitor at a corresponding position in the virtual world.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2019-082184 filed Apr. 23, 2019, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND

The present disclosure relates to an image generation device configuredto generate an image to be displayed on a head-mounted display, to animage display system, and to an information presentation method employedthereby.

Image display systems for watching a target space from a freely-selectedpoint of view are widely used. For example, image display systems aredeveloped to display panoramic video in accordance with a gaze directionof a user wearing a head-mounted display and rotating a user's head.Using a head-mounted display makes it possible to enhance the sense ofimmersion into video and improve the operability of applications such asgames. Further, some attractions are implemented for practicalapplications in order to allow a user wearing a head-mounted display toexperience free-roam virtual reality, which enables the user tovirtually walk around in a space displayed as a video image when theuser physically moves.

SUMMARY

When a user wears a non-transmissive head-mounted display, which blocksall outside light, it is obvious that the user is unable to experiencethe state of the outside world. When, for example, a free-roamattraction is used, it is possible to recognize initially existing itemsand other players in a play area due to virtual world settings. However,it is difficult to recognize an unexpected situation such as equipmentfailure in the play area or poor player health. If, for example, a staffmember enters the play area in an attempt to cope with such anunexpected situation, the staff member becomes an obstacle to increasethe level of danger.

The present disclosure has been made in view of the above circumstances,and it is desirable to provide a technology for increasing the level ofsafety during a movement of a user wearing a head-mounted display.

According to an embodiment of the present disclosure, there is providedan image generation device including: a space construction section thatconstructs a virtual world to be displayed on a head-mounted display; animage generation section that generates, based on a position and aposture of a head of a player wearing the head-mounted display, adisplay image representing the virtual world in a field of viewcorresponding to a point of view of the player, and causes thehead-mounted display to display the generated display image; and avisitor information acquisition section that acquires informationregarding presence of a visitor without a head-mounted display in aspace where the player is able to move. While the visitor is present,the space construction section displays an object indicating thepresence of the visitor at a corresponding position in the virtualworld.

According to another embodiment of the present disclosure, there isprovided an image display system including an image generation device, ahead-mounted display, and a sensor. The image generation deviceincludes: a space construction section that constructs a virtual worldto be displayed on the head-mounted display; an image generation sectionthat generates, based on a position and a posture of a head of a playerwearing the head-mounted display, a display image representing thevirtual world in a field of view corresponding to a point of view of theplayer, and causes the head-mounted display to display the generateddisplay image; and a visitor information acquisition section thatacquires information regarding presence of a visitor without ahead-mounted display in a space where the player is able to move. Whilethe visitor is present, the space construction section displays anobject indicating the presence of the visitor at a correspondingposition in the virtual world. The head-mounted display displays animage outputted from the image generation device. The sensor detects thevisitor and transmits data on the visitor to the image generationdevice.

According to a further embodiment of the present disclosure, there isprovided an image display system including an image generation device, ahead-mounted display, and an input device. The image generation deviceincludes: a space construction section that constructs a virtual worldto be displayed on a head-mounted display; an image generation sectionthat generates, based on a position and a posture of a head of a playerwearing the head-mounted display, a display image representing thevirtual world in a field of view corresponding to a point of view of theplayer, and causes the head-mounted display to display the generateddisplay image; and a visitor information acquisition section thatacquires information regarding presence of a visitor without ahead-mounted display in a space where the player is able to move. Whilethe visitor is present, the space construction section displays anobject indicating the presence of the visitor at a correspondingposition in the virtual world. The visitor information acquisitionsection detects the visitor's entry into and exit from a stay areapreset in the space. While the visitor is present in the stay area, thespace construction section displays an object representing a regioncorresponding to the stay area as the object indicating the presence ofthe visitor. The head-mounted display displays an image outputted fromthe image generation device. The input device inputs an operationindicative of the visitor's entry into and exit from the stay area, andtransmits information regarding the visitor's entry into and exit fromthe stay area to the image generation device.

According to a yet further embodiment of the present disclosure, thereis provided an information presentation method of an image generationdevice, including:

-   -   constructing a virtual world to be displayed on a head-mounted        display; generating, based on a position and a posture of a head        of a player wearing the head-mounted display, a display image        representing the virtual world in a field of view corresponding        to a point of view of the player, and causing the head-mounted        display to display the generated display image; acquiring        information regarding presence of a visitor without a        head-mounted display in a space where the player is able to        move; and displaying an object indicating the presence of the        visitor at a corresponding position in the virtual world while        the visitor is present.

Any combinations of the foregoing constituent elements and anyconversions of expressions of the embodiments of the present disclosure,for example, between methods, devices, systems, computer programs, datastructures, and recording media are also valid modes of the presentdisclosure.

According to the embodiments of the present disclosure, it is possibleto increase the level of safety during a movement of a user wearing ahead-mounted display.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example external view of a head-mounted display accordingto an embodiment of the present disclosure;

FIG. 2 is a diagram illustrating a configuration of a basic systemaccording to the embodiment;

FIG. 3 is a diagram illustrating an example of an image world that is tobe displayed on the head-mounted display by an image generation devicein the embodiment;

FIG. 4 is a schematic diagram illustrating a play area in a free-roamexhibit to which the embodiment is applicable;

FIG. 5 is a diagram illustrating an internal circuit configuration ofthe image generation device according to the embodiment;

FIG. 6 is a diagram illustrating a configuration of functional blocks ofthe image generation device in the embodiment;

FIGS. 7A and 7B are diagrams illustrating an example of change that aspace construction section in the embodiment applies to a virtual worldin accordance with the presence of a visitor;

FIGS. 8A and 8B are diagrams illustrating examples of display imagesthat represent the virtual world depicted in FIGS. 7A and 7B in thefield of view of a player with an image generation section;

FIGS. 9A and 9B are diagrams illustrating another example of change thatthe space construction section in the embodiment applies to a virtualworld in accordance with the presence of a visitor;

FIGS. 10A and 10B are diagrams illustrating examples of display imagesthat represent the virtual world depicted in FIGS. 9A and 9B in thefield of view of a player with the image generation section;

FIG. 11 is a flowchart illustrating processing that is performed by theimage generation device according to the embodiment in order to generatea display image and display the generated display image on thehead-mounted display; and

FIGS. 12A and 12B are diagrams illustrating examples of display imagesthat the image generation section generates in a case where a stay areaset in accordance with the embodiment is used as a player's shelter.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 illustrates an example external view of a head-mounted display100 according to an embodiment of the present disclosure. Thehead-mounted display 100 includes a main body section 110, a foreheadcontact section 120, and a temporal region contact section 130. Thehead-mounted display 100 is a display device that is worn on the head ofa user to allow the user to watch, for example, still images and videoimages displayed on a display and listen to, for example, sound andmusic outputted from headphones. A motion sensor built in or externallyattached to the head-mounted display 100 is able to make measurements toobtain posture information such as a rotation angle and tilt of the headof the user wearing the head-mounted display 100. Therefore, changingthe field of view of a display image in accordance with a motion of theuser's head causes the user to feel like as if the user is in an imageworld.

However, the shape and structure of the head-mounted display accordingto the present embodiment of the present disclosure are not limited tothose depicted in FIG. 1 . For example, a camera may be attached to afront surface of the head-mounted display 100 in order to capture animage of a real space as viewed from a user's face. In such a case,analyzing the captured image makes it possible to acquire an orientationof the user's face and a position of the user's head and determine aposition of a real item existing ahead. The captured image can also beused as the display image.

FIG. 2 is a diagram illustrating a configuration of a basic systemaccording to the present embodiment. The head-mounted display 100 isconnected to an image generation device 200 through an interface 205that is to be connected to a peripheral such as a wireless communicationor a universal serial bus (USB) device. The image generation device 200may be further connected to a server through a network. In such a case,the server may supply, to the image generation device 200, an onlineapplication such as a game in which a plurality of users can participatethrough the network.

The image generation device 200 determines a position of a virtual pointof view and a direction of a gaze with respect to the display image inaccordance with aa head position and a posture of the user wearing thehead-mounted display 100, generates the display image so as to provide acorresponding field of view, and outputs the generated display image tothe head-mounted display 100. As far as the image generation device 200performs the above operations, it may display images for any appropriatepurposes. For example, the image generation device 200 may generate adisplay image representing a virtual world that functions as a stage ofan electronic game while it progresses, or display still images andvideo images for appreciation purposes.

Displaying an omnidirectional (360-degree) image so as to cover a wideangular range centered on a virtual point of view gives a sense ofimmersion into the image world. The image generation device 200 may be aseparate device connected to the head-mounted display 100 through theinterface 205 as depicted in FIG. 2 or may be incorporated integrallyinto the head-mounted display 100.

FIG. 3 is a diagram illustrating an example of the image world that isto be displayed on the head-mounted display 100 by the image generationdevice 200 in the present embodiment. This example represents a statewhere a user 12 is in a room that is a virtual world. Objects, such aswalls, a floor, a window, a table, and items on the table, are disposedin a world coordinate system defining the virtual world, as depicted inFIG. 3 . The image generation device 200 defines a view screen 14 inaccordance with the position of the point of view of the user 12 and thegaze direction of the user 12 in the world coordinate system, and drawsthe display image by projecting images of the objects onto the viewscreen 14.

When the position of the point of view of the user 12 and the gazedirection of the user 12 (hereinafter these may be comprehensivelyreferred to as the “point of view”) are acquired at a predetermined rateand a position and an orientation of the view screen 14 are changedaccordingly, images can be displayed in a field of view corresponding tothe user's point of view. When stereo images having parallax aregenerated and displayed in front of the left and right eyes of the user12 by using the head-mounted display 100, a virtual space can also beviewed three-dimensionally. This enables the user 12 to experiencevirtual reality as if the user 12 is immersed in a room in a displayworld. The illustrated example assumes that a virtual world based oncomputer graphics is regarded as a display target. However, the displaytarget may be an image that is obtained by combining computer graphicswith a captured image of a real world such as a panoramic photograph.

The above-described technology enables the user 12 to freely walk aroundin a virtual world. When, for example, a position and a posture ofanother user existing in the same space as the user 12 are acquired todispose an object representing an avatar of the other user in a virtualspace, the user 12 visually recognizes the other user as a part of thevirtual world. This scheme is used to actually utilize a free-roammultiplayer virtual experience attraction. The free-roam multiplayervirtual experience attraction (hereinafter referred to as the “free-roamexhibit”) allows a plurality of players to fight and explore together ina virtual space.

FIG. 4 schematically illustrates a play area in the free-roam exhibit towhich the embodiment is applicable. A play area 20 is illustrated so asto overlook a real space where the free-roam exhibit is presented.Players 10 a to 10 d wearing head-mounted displays 100 a to 100 d,respectively, exist in the play area 20. However, the number of playersis not limited. The image generation device 200 constructs a virtualworld as depicted in FIG. 3 , and causes the head-mounted displays 100 ato 100 d worn by the players 10 a to 10 d to display the virtual worldin a field of view corresponding to the point of view of each player.

The position and posture of each of the players 10 a to 10 d areacquired, for example, by obtaining values measured by motion sensorsbuilt in the head-mounted displays 100 a to 100 d, acquiring imagescaptured by cameras included in the head-mounted displays 100 a to 100d, and analyzing the measured values and captured images. Informationobtained in this manner can also be used for constructing a virtualworld. More specifically, respective objects indicating the movement ofthe players 10 a to 10 d are disposed in a virtual world in order to letthe players recognize the presence and movement of the other players.

Basically, in the above-described free-roam exhibit, persons other thanthe players 10 a to 10 d are not expected to enter the play area.However, it is quite likely that a staff member or a person other thanthe players is compelled to enter the play area in an attempt to copewith various severities of an unexpected situation such as equipmentfailure, abnormal physical conditions or injuries of the players, andthe loss of personal belongings of the players. In such a situation, theplayers 10 a to 10 d viewing only a virtual world including the otherplayers feel the presence of an invisible item and thus becomedistracted from careful watching or cause an increased risk ofcollision. Further, it is not practical to interrupt all plays forsafety's sake each time a minor incident occurs.

In view of the above circumstances, the present embodiment separatelyincludes a mechanism for detecting visitors other than the players, andcauses images viewed by the players 10 a to 10 d to reflect the presenceof such a visitor. A person who is not a player but enters the play areais hereinafter referred to as a “visitor.” The visitor is a person whodoes not wear the head-mounted display 100, that is, a person who sees areal space. The visitor is not to be tracked in an ordinary free-roamsystem for tracking the position and posture of a player. As far as thevisitor is such a person, the purpose of the visitor's entry into theplay area and the attribute of the visitor are not specifically limited.

In order to implement the above features, an image display systemaccording to the present embodiment incorporates a sensor 22 and aninput device 24 in addition to the image generation device 200 and thehead-mounted displays 100 a to 100 d. The sensor 22 tracks a visitor 16.The input device 24 inputs an operation that is performed to reflect thepresence of the visitor 16 in a display image. For example, the visitor16 wears a marker and a device 26 for generating an ultrasonic wave orlight having a predetermined wavelength band, and the sensor 22 tracksthe presence and position of the visitor 16 by detecting the ultrasonicwave or the light having the predetermined wavelength band.

Based on the result of such tracking, the image generation device 200disposes an object representing the visitor 16 at a correspondingposition in a virtual world viewed by the players 10 a to 10 d.Obviously, the object in the virtual world is moved while the visitor 16is moving. When rules are established to define a position at which thevisitor 16 wears the device 26, such as a forward position (a positiontoward which the visitor faces) of the visitor's head or higher than thevisitor's head, or a front surface of the visitor's body, the directionin which the visitor 16 moves can be predicted based on the orientationof the face and body of the visitor 16.

A headgear with the device 26 attached to its front surface may becreated and worn by the visitor 16 entering the play area. Analternative is to prepare a mounting fixture with a bar, attach thedevice 26 to the tip of the bar, allow the visitor 16 to wear themounting fixture when the visitor 16 enters the play area, and fix thedevice 26 at a position above the head of the visitor 16. Raising theposition of the device 26 reduces the possibility of the device 26 beinghidden by the visitor 16 and the players 10 a to 10 d and thus enablesthe sensor 22 to achieve stable detection.

The image generation device 200 may allow the players 10 a to 10 d todetermine a direction of travel of the visitor 16 by labeling an objectrepresentative of the visitor 16 with the direction of travel of thevisitor 16. This enables the players 10 a to 10 d to intuitively sensethe presence of the visitor 16 and naturally act without colliding withthe visitor 16. The shape and position of the sensor 22 are not limitedto those depicted. The sensor 22 may be mounted, for example, on aceiling to overlook the interior of a room or may be movable.Alternatively, the sensor 22 may be built in the image generation device200.

The input device 24 may be a common input device such as a remotecontroller, a game controller, a mobile terminal, a pushbutton, or ajoystick. The input device 24 inputs an operation that determines a timepoint at which the presence of the visitor 16 is reflected in imagesviewed by the players 10 a to 10 d. For example, a stay area 28 of thevisitor 16 is clearly marked in the play area 20 in advance, and thevisitor 16 entering the stay area 28 operates the input device 24 toinput information indicating that the visitor 16 has entered the stayarea 28. The input device 24 then receives the inputted information andtransmits the received inputted information to the image generationdevice 200. The image generation device 200 then causes an object toappear in a region within a virtual world viewed by the players 10 a to10 d that corresponds to the stay area 28.

As a result, the players 10 a to 10 d are able to act without enteringthe stay area 28. Here, the extent, the shape, and the number of thestay area 28 are not limited. For example, the stay area 28 may be, asillustrated, an island type not in contact with the end of the play area20 or a region in a corner of the play area 20. Alternatively, the stayarea 28 may be a passage type that crosses the play area 20. Further,the periphery of the stay area 28 is marked by a luminescent marker orby a periphery-indicating tape attached to a floor in order to let thevisitor 16 understand the periphery of the stay area 28 in the realworld.

The input device 24 may be operated not only by the visitor 16 but alsoby, for example, an administrator present outside of the play area 20.The administrator may watch, for example, a surveillance monitor toconfirm the entry of the visitor 16 into the stay area 28 and thenoperate the input device 24. Further, when the visitor 16 exits the stayarea 28, the visitor 16 operates the input device 24 to inputinformation indicative of the exit. This causes the image generationdevice 200 to remove an object representing the stay area 28 from thevirtual world.

Introducing the sensor 22 allows the visitor 16 to freely move in theplay area 20. Meanwhile, if the place to stay is limited, the visitor 16should avoid the players, enter the stay area 28, and operate the inputdevice 24. As described above, only one of the sensor 22 and the inputdevice 24 may be introduced in accordance with the action to be taken bythe visitor 16.

An alternative is to track a moving visitor 16 by using the sensor 22,reflect the movement of the visitor 16 in an object serving as avisitor's avatar, detect the entry of the visitor 16 into the stay area28 by using the sensor 22 or the input device 24, and switch anexpression to an object representing the stay area. In this case, due tothe operation of only the sensor or the collaboration between the sensor22 and the input device 24, an object indicating the presence of thevisitor is continually expressed in different forms. In any case, theimage generation device 200 is wiredly or wirelessly connected to boththe sensor 22 and the input device 24, and acquires, as needed,information transmitted from both of them.

FIG. 5 illustrates an internal circuit configuration of the imagegeneration device 200. The image generation device 200 includes acentral processing unit (CPU) 222, a graphics processing unit (GPU) 224,and a main memory 226. These constituent elements are interconnectedthrough a bus 230. The bus 230 is further connected to an input/outputinterface 228.

The input/output interface 228 is connected to a communication section232, a storage section 234, an output section 236, an input section 238,and a recording medium drive section 240. The communication section 232includes USB, IEEE 1394 (IEEE: Institute of Electrical and ElectronicsEngineers), and other peripheral device interfaces and a wired orwireless local area network (LAN) interface. The storage section 234 is,for example, a hard disk drive or a nonvolatile memory. The outputsection 236 outputs data to the head-mounted display 100. The inputsection 238 inputs data from the head-mounted display 100, the sensor22, and the input device 24. The recording medium drive section 240drives a removable recording medium such as a magnetic disk, an opticaldisc, or a semiconductor memory.

When an operating system stored in the storage section 234 is executed,the CPU 222 exercises overall control of the image generation device200. Further, the CPU 222 executes various programs that are read from aremovable recording medium and loaded into the main memory 226 ordownloaded through the communication section 232. The GPU 224 has afunction of a geometry engine and a function of a rendering processor,performs drawing processing in accordance with a drawing command fromthe CPU 222, and stores the display image in an undepicted frame buffer.The GPU 224 then converts the display image stored in the frame bufferto a video signal and outputs the video signal to the output section236. The main memory 226 includes a random-access memory (RAM) andstores the programs and data to be used for processing.

FIG. 6 illustrates a configuration of functional blocks of the imagegeneration device 200 in the present embodiment. As mentioned earlier,the image generation device 200 may perform general informationprocessing such as causing an electronic game to progress orcommunicating with a server. However, FIG. 6 particularly focuses on afunction of controlling the display image in accordance with a visitor'sentry and exit. At least some of the functions of the image generationdevice 200, which are depicted in FIG. 6 , may be implemented in thehead-mounted display 100. Alternatively, at least some of the functionsof the image generation device 200 may be implemented in a server thatis connected to the image generation device 200 through a network.

Further, the functional blocks depicted in FIG. 6 can be implemented byhardware such as the CPU, GPU, memories, or other constituent elementsdepicted in FIG. 5 , or implemented by software including programs forexercising various functions such as a data input function, a dataretention function, an image processing function, and a communicationfunction, the programs being loaded from a recording medium or the likeinto a memory. Therefore, it will be understood by those skilled in theart that the functional blocks may be implemented by hardware alone, bysoftware alone, or by a combination of hardware and software and are notto be limited to any of them.

The image generation device 200 includes a player position informationacquisition section 50, a point-of-view information acquisition section52, a visitor information acquisition section 58, a space constructionsection 60, an image generation section 54, an image data storagesection 62, and an output section 56. The player position informationacquisition section 50 acquires information regarding the position of aplayer in the play area. The point-of-view information acquisitionsection 52 acquires information regarding the point of view of a user.The visitor information acquisition section 58 acquires informationregarding the entry, exit, and position of a visitor. The spaceconstruction section 60 constructs a virtual world. The image generationsection 54 generates the display image. The image data storage section62 stores image data to be used for generating the display image. Theoutput section 56 outputs the display image to the head-mounted display100.

The player position information acquisition section 50 includes, forexample, the input section 238 and the CPU 222, which are depicted inFIG. 5 , and acquires information regarding the position of each playerin the play area at a predetermined rate. The information regarding theposition of each player can be obtained by successively acquiring outputdata from various motion sensors and cameras built in the head-mounteddisplay 100 worn by each player as mentioned earlier and analyzing theacquired output data.

Alternatively, for example, a camera separately disposed in the playarea may be used to perform image analysis in order to track theposition and posture of a player wearing a luminescent marker of apredetermined color. Another alternative is to attach a deviceconfigured to generate, for example, an ultrasonic wave detectable bythe sensor 22 to the body of a player. When such an alternative is used,the player position information acquisition section 50 acquires datatransmitted from the sensor 22, and determines the position and postureof the player in accordance with the acquired data.

Further, the player position information acquisition section 50internally retains data defining a normal range of a parameterindicating the posture of a player, and detects a state where thedetermined posture of the player is outside the normal range. In a casewhere an ultrasonic wave or light from, for example, a marker worn by aplayer is detected in order to determine the posture of the player, itis conceivable that the result of player detection may be affected by anapproaching visitor. Accordingly, the posture determined based on theresult of player detection may be in an abnormal state different from anactual state.

Consequently, the player position information acquisition section 50detects such a state, and prevents the detected state from beingreflected in the posture of an object representing the player. Dataindicating, for example, postures assumable at a time by a human and arange of possible movement of a human during a limited period of time isprepared as the data defining the normal range. As regards the dataindicating the range of possible movement of a human during a limitedperiod of time, the player position information acquisition section 50determines, based on posture changes from a previous time step, whetheror not the normal range is exceeded.

The point-of-view information acquisition section 52 includes, forexample, the input section 238 and the CPU 222, which are depicted inFIG. 5 , and acquires information regarding the position of eachplayer's point of view and the gaze direction of each player at apredetermined rate. For example, the point-of-view informationacquisition section 52 successively acquires output values from anacceleration sensor built in the head-mounted display 100, and acquiresthe posture of a player's head in accordance with the acquired outputvalues. The position of the player's head in a real space may beacquired by analyzing an image of a luminescent marker attached to thehead-mounted display 100, which is captured by a camera separatelydisposed in the play area as mentioned earlier.

Alternatively, the position and posture of the head of a player may beacquired by analyzing an image captured by a camera provided for thehead-mounted display 100 through the use, for example, of a simultaneouslocalization and mapping (SLAM) technology. When the position andposture of the player's head are acquired, it is possible to roughlydetermine the position of the player's point of view and the gazedirection of the player. Further, it will be understood by those skilledin the art that the information regarding the player's point of view canbe acquired by various methods. As the processing performed by theplayer position information acquisition section 50 is similar to theprocessing performed by the point-of-view information acquisitionsection 52, the result of processing performed by the point-of-viewinformation acquisition section 52 may be shared by the player positioninformation acquisition section 50.

The visitor information acquisition section 58 includes, for example,the input section 238 and the CPU 222, which are depicted in FIG. 5 ,and acquires information regarding the presence of a visitor other thanplayers. More specifically, based on data transmitted from the sensor22, the visitor information acquisition section 58 detects the entry andexit of a visitor, and acquires the position of the visitor in the playarea at a predetermined rate. In some cases, based on data transmittedfrom the input device 24, the visitor information acquisition section 58detects a visitor's entry into or exit from the stay area previously setin the play area. As mentioned earlier, the visitor informationacquisition section 58 may acquire data from one of or both the sensor22 and the input device 24.

The space construction section 60 includes, for example, the CPU 222depicted in FIG. 5 , and constructs the space of a virtual world to bedisplayed on the head-mounted display 100. The details of the space tobe constructed vary with the purpose of a free-roam exhibit. In anycase, the space construction section 60 disposes an object representinga player at a virtual location to be displayed. Players in the virtualworld are allowed to visually recognize each other by moving individualobjects in accordance with the result of player tracking, which isacquired by the player position information acquisition section 50. In acase where the player position information acquisition section 50acquires the player's posture as well, the space construction section 60changes an object's posture as well in accordance with the player'sposture.

If it is detected that the calculated posture of a player, which isdetermined by the player position information acquisition section 50, isoutside the normal range, the space construction section 60 locks theposture of the corresponding object in an immediately preceding state.In this instance, the space construction section 60 changes a mode ofthe locked object, for example, by making the locked objectsemitransparent or changing the color of the locked object. The displayimage then causes the other players to understand that temporarymeasures are taken. When it is detected that the calculated posture hasreturned to the normal range, the space construction section 60 restoresthe corresponding object to its previous mode and causes the prevailingposture to be reflected in the corresponding object.

Further, while a visitor is in the play area, the space constructionsection 60 displays an object indicating the presence of the visitor ata corresponding position in the virtual world. More specifically, whenthe position of the visitor is being tracked by the sensor 22, the spaceconstruction section 60 displays an object that moves in accordance withthe movement of the visitor. In such an instance, the space constructionsection 60 may determine an orientation of the face and body of thevisitor in accordance with the result of tracking by the sensor 22, andlabel the corresponding object with a direction of travel inferred fromthe determined orientation. For example, the object may be affixed witha pictorial figure indicating the direction of travel. As analternative, a face of the object may be created to indicate thedirection of travel. The space construction section 60 may predict animmediately succeeding direction of travel in accordance with a recordof previous movements of the visitor, and causes the result ofprediction to be reflected in the object.

Alternatively, in a case where a visitor's entry into and exit from thestay area are detected based on data from the input device 24 or thesensor 22, the space construction section 60 displays an objectrepresenting the stay area while the visitor is in the stay area. Theimage data storage section 62 is implemented, for example, by the mainmemory 226 depicted in FIG. 5 , and stores three-dimensional models, forexample, of the virtual world to be constructed by the spaceconstruction section 60, player objects, and visitor objects.

The image generation section 54 is implemented, for example, by the CPU222, the GPU 224, and the main memory 226, which are depicted in FIG. 5, and generates a display image representative of a virtual world at apredetermined rate in a field of view corresponding to the point of viewof each player, which is acquired by the point-of-view informationacquisition section 52. Stated differently, the image generation section54 sets a view screen so as to match the point of view of each player,and presents a display image by projecting the virtual world constructedby the space construction section 60 onto the view screen. In a casewhere stereo images are used to provide a stereoscopic vision, the imagegeneration section 54 generates left- and right-eye images by settingthe view screen for each of the left and right eyes.

The output section 56 includes, for example, the CPU 222, the mainmemory 226, and the output section 236, which are depicted in FIG. 5 ,and transmits display image data generated by the image generationsection 54 to the head-mounted display 100 at a predetermined rate. In acase where the image generation section 54 generates stereo images toprovide a stereoscopic vision, the output section 56 generates a displayimage by joining the generated stereo images as left and right stereoimages, and outputs the generated display image. In a case where thehead-mounted display 100 is configured for watching a display imagethrough a lens, the output section 56 corrects the display image inconsideration of distortion caused by the lens.

FIGS. 7A and 7B illustrate an example of change that the spaceconstruction section 60 applies to a virtual world in accordance withthe presence of a visitor. Depicted in FIG. 7A are a real space in whichno visitor is in a play area and a virtual world corresponding to thereal space. Players 10 exist in the play area 20, which is the realspace. Meanwhile, in a virtual world 30 a constructed by the spaceconstruction section 60, objects 32 representing the respective players10 are disposed at positions corresponding to those of the players 10.

The space construction section 60 moves the objects 32 in accordancewith the movement of the players 10. For example, an object 34representing a character other than the players 10 is disposed as neededin the virtual world 30 a. If, in such a situation, a visitor 16 entersthe play area 20 as depicted in FIG. 7B, the movement of the visitor 16is tracked by the sensor 22. The space construction section 60 thencauses an object 36 to appear in a virtual world 30 b. The object 36moves in accordance with the movement of the visitor 16.

When a device configured to generate, for example, an ultrasonic wave isattached to the front surface of the body or head of the visitor 16, itis possible to infer the direction of travel of the visitor 16.Therefore, the space construction section 60 attaches a direction object38 indicative of the direction of travel to the object 36 representingthe visitor. As mentioned earlier, the direction of travel may beinferred from the track of movement. Further, in the present example,the direction object 38 is shaped like an arrow indicative of thedirection of travel. However, the shape of the direction object 38 isnot specifically limited as far as the players are able to recognize thedirection. Furthermore, the shapes of the objects representing thevisitor or players are not specifically limited. When the object 36representing the visitor is shaped like a human, the direction of travelof the visitor may be indicated by the orientation of the face of thevisitor.

FIGS. 8A and 8B illustrate examples of display images that represent thevirtual world depicted in FIGS. 7A and 7B in the field of view of aplayer with the image generation section 54. Depicted in FIG. 8A is astate where no visitor exists in the play area as depicted in FIG. 7A,and the objects 32 representing the other players are visible in thevirtual world. Depicted in FIG. 8B is a state where a visitor exists inthe play area as depicted in FIG. 7B, and the objects 32 representingthe other players and the object 36 representing the visitor are bothvisible. When the above-described images are displayed, the players areable to intuitively sense the direction in which the visitor is presentand the direction of travel of the visitor, and thus do not feeluncomfortable with or collide with an invisible presence. Further, thesense of immersion will not be reduced because such an invisiblepresence is recognized as a part of the virtual world.

FIGS. 9A and 9B illustrate another example of change that the spaceconstruction section 60 applies to a virtual world in accordance withthe presence of a visitor. Depicted in FIG. 9A are a real space in whichno visitor is in the play area and a virtual world corresponding to thereal space. Players 10 exist in the play area 20, which is a real space.In the play area 20 depicted in the present example, the stay area 28 ofa visitor is labeled, for example, with a marker or a tape. Meanwhile,in a virtual world 40 a constructed by the space construction section60, objects 32 representing the respective players 10 and an object 34representing a character other than the players are disposed, as is thecase with FIG. 7A.

In the above case, too, the space construction section 60 moves theobjects 32 in accordance with the movement of the players 10. However,in a state where no visitor exists as illustrated in FIG. 9A, the stayarea 28 does not exist in the virtual world 40 a, either. If, in such asituation, the visitor 16 enters the stay area 28 as depicted in FIG. 9Band is detected due, for instance, to an operation of the input device24, the space construction section 60 causes an object 42 representing aregion corresponding to the stay area 28 to appear in a virtual world 40b. In the illustrated example, the object 42 is shaped like a fencesurrounding the region. However, the shape of the object 42 is notspecifically limited as far as the players are able to recognize thatthey are not allowed to enter the region.

FIGS. 10A and 10B illustrate examples of display images that representthe virtual world depicted in FIGS. 9A and 9B in the field of view of aplayer with the image generation section 54. Depicted in FIG. 10A is astate where no visitor is in the play area as depicted in FIG. 9A, andthe objects 32 representing the other players are visible in the virtualworld. Meanwhile, the image depicted in FIG. 10A does not permit thestay area to be visually recognized. Even if a visitor is present in theplay area, the same image may be displayed as far as the visitor is notin the stay area. Alternatively, based on the result of tracking by thesensor 22, the object 36 moving in accordance with the movement of thevisitor may be displayed as depicted in FIG. 8B.

Depicted in FIG. 10B is a state where a visitor is in the stay area asdepicted in FIG. 9B, and the objects 32 representing the other playersand the object 42 representing a region corresponding to the stay areaare both visible. When the above-described images are displayed, theplayers are able to intuitively sense a region that the players are notallowed to enter, and thus do not feel uncomfortable with or collidewith an invisible presence. Further, in this case, too, the sense ofimmersion will not be reduced because such an invisible presence isrecognized as a part of the virtual world.

An operation of the image generation device 200, which is implemented bythe above-described configuration, will now be described. FIG. 11 is aflowchart illustrating processing that is performed by the imagegeneration device 200 according to the present embodiment in order togenerate a display image and display the generated display image on thehead-mounted display 100. The processing depicted in the flowchartstarts in a state where a player wears the head-mounted display 100 andthe image generation device 200 establishes communication with thehead-mounted display 100 and the sensor 22 or with the input device 24.In this state, the head-mounted display 100 transmits, for example, thevalues measured by the motion sensors to the image generation device200.

The space construction section 60 then starts generating a virtual worldthat is to be displayed (step S10). In a case where there are aplurality of players, objects representing the players are included inthe virtual world. Subsequently, the space construction section 60performs a process of moving the objects in accordance with thepositions of the players while the processing steps in FIG. 11 areperformed. The objects may be moved in accordance with the movement ofthe players. In this instance, when the posture of a player subjected toa tracking process is outside the normal range, the space constructionsection 60 applies image effects, for example, by locking the posture ofa corresponding object in an immediately preceding state and making thelocked object semitransparent. When it is determined that the posturehas returned to the normal range, the space construction section 60restores the object to its previous mode.

If no visitor entering the play area or the stay area is detected (“N”at step S12), the point-of-view information acquisition section 52acquires the point-of-view information regarding each player (step S16),and the image generation section 54 generates a display imagerepresenting a virtual world in a field of view corresponding to thepoint of view of each player and thus causes the output section 56 tooutput data on the display image (step S18). When the outputted data isdisplayed on the head-mounted display 100 of each player, the virtualworld depicted, for example, in FIG. 8A or FIG. 10A is visuallyrecognized.

As far as no visitor is present (“N” at step S20) and it is notnecessary to end an image display operation (“N” at step S26), thedisplay image from the point of view of each player is successivelygenerated and outputted (“N” at step S12, step S16, and step S18). If avisitor entering the play area or the stay area is detected (“Y” at stepS12), the space construction section 60 adds an object indicating thepresence of the detected visitor to the virtual world (step S14).

More specifically, in a case where the position of a visitor is beingtracked, an object indicating the movement of the visitor is added tothe virtual world. When the entry of the visitor into the stay area isdetected, an object representing a region corresponding to the stay areais added. When, in this state, the point-of-view information acquisitionsection 52 acquires the point-of-view information regarding each player(step S16) and the image generation section 54 generates a display imagecorresponding to the acquired point-of-view information, the outputsection 56 outputs data on the generated display image (step S18).

When the generated display image is displayed on the head-mounteddisplay 100 of each player, the presence of the visitor is visuallyrecognized as depicted in FIG. 8B and FIG. 10B. As far as the visitordoes not exit (“N” at step S20) and it is not necessary to end the imagedisplay operation (“N” at step S26), the display image from the point ofview of each player is successively generated and outputted (“N” at stepS12, step S16, and step S18). However, if a new visitor enters the playarea or enters another stay area (“Y” at step S12), an object indicatingthe presence of the new visitor is further added to the virtual world(step S14).

Meanwhile, if it is detected that a visitor in the play area or the stayarea has exited (“Y” at step 20), the space construction section 60removes an object indicating the presence of the visitor from thevirtual world (step S24). As far as it is not necessary to end the imagedisplay operation (“N” at step S26), steps S12 to S24 are repeated bybranching as needed between relevant steps. When it becomes necessary toend the image display operation (“Y” at step S26), the whole processingends.

The stay area in the present embodiment may be used not only as avisitor's shelter but also as a player's shelter. More specifically, aplayer in a poor physical condition or feeling tired in the middle of aplay may use the stay area as a shelter where the player is notinterfered with by another player. As mentioned earlier, the stay arealabeled within the play area is not visually recognized by the playersunder normal conditions. Therefore, the space construction section 60 orthe image generation section 54 operates in such a manner that apictorial figure indicating a direction toward the stay area as viewedfrom a relevant player is depicted in the virtual world or in thedisplay image.

FIGS. 12A and 12B illustrate examples of display images that the imagegeneration section 54 generates in a case where the stay area set inaccordance with the embodiment is used as a player's shelter. Depictedin FIG. 12A is an image displayed on the head-mounted display 100 of aplayer who wants to take shelter in the stay area. When a player isplaced in a situation where the player wants to take shelter, the playerindicates such a situation by performing an input operation with anundepicted input device possessed by the player or with the head-mounteddisplay 100 that moves based on the motion of the head of the player.Accordingly, the image generation section 54 operates in such a mannerthat a pictorial FIG. 44 guiding the player to the stay area appearswithin the image to be displayed on the head-mounted display 100 of theplayer.

In the present example, the pictorial FIG. 44 is shaped like an arrowindicating a direction toward the stay area. However, the shape of thepictorial FIG. 44 is not specifically limited as far as the player isable to recognize the direction toward the stay area. The pictorial FIG.44 may be depicted in the virtual world by the space constructionsection 60, or may be superimposed on a displayed image by the imagegeneration section 54. The player position information acquisitionsection 50 detects that the player has moved in accordance with thepictorial FIG. 44 and entered the stay area. In such an instance, thespace construction section 60 causes an object representing a regioncorresponding to the stay area to appear in the virtual world, as is thecase with a visitor.

Depicted in FIG. 12B is an image that is displayed on the head-mounteddisplay 100 of another player in the resulting situation. In thisinstance, too, the object 42 representing a region corresponding to thestay area is displayed as is the case with FIG. 10B. This enables theother player to intuitively sense a region that cannot be entered, andthus avoid interfering with a sheltered player. The image displayed onthe head-mounted display 100 of the sheltered player is changed in apredetermined manner at the time point of entry into the stay area inorder to notify the sheltered player of the entry into the stay area.

Subsequently, if, for example, a visitor enters, as needed, the samestay area as the sheltered player and then exits together with thesheltered player, and such a situation is detected by the playerposition information acquisition section 50 or the visitor informationacquisition section 58, the space construction section 60 removes theobject 42 so as to return to a normal state. Alternatively, when aproblem encountered by the sheltered player is solved, the shelteredplayer is able to exit the stay area and return to a play. In such acase, too, the space construction section 60 removes the object 42 fromthe virtual world at a time point at which the sheltered player exitsthe stay area.

In a free-roam exhibit where a player uses a head-mounted display towalk around in a virtual world, the present embodiment, which has beendescribed above, detects a person other than the player has entered aplay area and causes the presence of such a detected person to bereflected in an image displayed on the head-mounted display of theplayer. More specifically, the present embodiment tracks the position ofa visitor by attaching a sensor-detectable device to the visitor andallowing a sensor to detect the sensor-detectable device. In this case,the present embodiment causes an object representing the visitor toappear in the virtual world.

An alternative is to set a dedicated stay area in the play area and use,for example, an input device to detect, for instance, a visitor enteringthe dedicated stay area. In this case, the present embodiment causes anobject representing a region corresponding to the dedicated stay area toappear while the visitor is present in the stay area. In either of theabove cases, the present embodiment enables even a player viewing onlythe virtual world to intuitively sense the presence of a visitor andprevents the player from feeling uncomfortable with the intuitivelysensed presence of the visitor and from colliding with the visitor.

Further, the present embodiment is able to recognize even such a suddenevent as a part of the virtual world. This enables the player tocontinue playing without losing the sense of immersion. Furthermore,when a player having a problem during a play is guided to the stay areaand an object representing the player is made to appear in a similarmanner, the present embodiment enables the player to address the problemwithout interfering with another player. Moreover, when the result ofplayer detection is affected by a visitor and a calculated posture isdetected to be abnormal, the present embodiment prevents the calculatedposture from being reflected in a corresponding object. This makes itpossible to minimize the influence of the presence of a visitor on aplay.

While the present disclosure has been described in conjunction with anembodiment, it will be understood by those skilled in the art that theembodiment is illustrative and not restrictive, and that the combinationof constituent elements and individual processes in the embodiment maybe variously modified, and further that such modifications also fallwithin the scope of the present disclosure.

What is claimed is:
 1. An image generation device comprising: a spaceconstruction section that constructs a virtual world to be displayed ona head-mounted display; an image generation section that generates,based on a position and a posture of a head of a player wearing thehead-mounted display, a display image representing the virtual world ina field of view corresponding to a point of view of the player, andcauses the head-mounted display to display the generated display image;and an obstacle information acquisition section that acquiresinformation regarding presence of an obstacle which is not related tothe player in a space where the player is located, wherein, while theobstacle is present within the space, the space construction sectiondisplays a first object in the virtual world corresponding to theobstacle.
 2. The image generation device according to claim 1, whereinthe obstacle is able to move in the space, wherein the object isdisplayed at a position in the virtual world corresponding to a locationof the obstacle in the space, and wherein, when the object is not withinthe space, the first object is not displayed.
 3. The image generationdevice according to claim 1, wherein the obstacle informationacquisition section acquires information regarding a position of theobstacle in accordance with a result of detection by a sensor, thesensor detecting an ultrasonic wave, the ultrasonic wave being generatedfrom a device worn by the obstacle, wherein the sensor is located at afixed position in the space.
 4. The image generation device according toclaim 1, wherein the obstacle information acquisition section acquiresinformation regarding a position of the obstacle in accordance with aresult of detection by a sensor, the sensor detecting light having apredetermined color, the light having a predetermined color beinggenerated from a device worn by the obstacle, wherein the sensor islocated at a fixed position in the space.
 5. The image generation deviceaccording to claim 1, wherein the obstacle information acquisitionsection predicts a direction of travel of the obstacle, and the spaceconstruction section displays a second object moving in addition to thefirst object in accordance with a movement of the obstacle as the objectindicating the presence of the obstacle, and labels the second objectwith the direction of travel.
 6. The image generation device accordingto claim 1, wherein the obstacle information acquisition section detectsthe obstacle's entry into and exit from a stay area preset in the space,while the obstacle is present in the stay area, the space constructionsection displays an object surrounding a region corresponding to thestay area as the object indicating the presence of the obstacle, andwherein the stay area is a sub-area of the space represented in thevirtual space at a corresponding location.
 7. The image generationdevice according to claim 6, wherein the obstacle informationacquisition section detects the obstacle's entry into and exit from thestay area through an input device operated by the obstacle or a personoutside the space.
 8. The image generation device according to claim 6,wherein, when an input operation is performed by the player to indicatethat the player wants to take shelter, the image generation sectiondisplays a pictorial figure for guiding the player to the stay area inthe display image on a head-mounted display worn by the player, and thespace construction section displays the object surrounding the regioncorresponding to the stay area even while the player is present in thestay area.
 9. The image generation device according to claim 1, furthercomprising: a player position information acquisition section thatacquires information regarding the position and the posture of theplayer, and detects a state where the posture indicated by the acquiredinformation is outside a predefined normal range, wherein the spaceconstruction section displays a second object representing the postureof the player at a corresponding position in the virtual world, and whenthe posture is outside the normal range, locks the second object andchanges a mode of the locked object, the posture of the player beingdetermined based on the information acquired by the player positioninformation acquisition section.
 10. An image display system comprising:an image generation device including a space construction section thatconstructs a virtual world to be displayed on a head-mounted display, animage generation section that generates, based on a position and aposture of a head of a player wearing the head-mounted display, adisplay image representing the virtual world in a field of viewcorresponding to a point of view of the player, and causes thehead-mounted display to display the generated display image, and anobstacle information acquisition section that acquires informationregarding presence of an obstacle which is not related to the player ina space where the player is able to move, while the obstacle is presentin the space, the space construction section displaying an object in thevirtual world corresponding to the obstacle, the head-mounted displaythat displays an image outputted from the image generation device; and asensor located at a fixed position that detects the obstacle andtransmits data on the obstacle to the image generation device.
 11. Animage display system comprising: an image generation device including aspace construction section that constructs a virtual world to bedisplayed on a head-mounted display, an image generation section thatgenerates, based on a position and a posture of a head of a playerwearing the head-mounted display, a display image representing thevirtual world in a field of view corresponding to a point of view of theplayer, and causes the head-mounted display to display the generateddisplay image, and an obstacle information acquisition section thatacquires information regarding presence of an obstacle which is notrelated to the player in a space where the player is able to move, whilethe obstacle is present, the space construction section displaying anobject in the virtual world corresponding to the obstacle, wherein theobstacle information acquisition section detects the obstacle's entryinto and exit from a stay area preset in the space, wherein the stayarea is a sub-area of the space represented in the virtual space at acorresponding location, while the obstacle is present in the stay area,the space construction section displaying an object surrounding a regioncorresponding to the stay area as the object indicating the presence ofthe obstacle; the head-mounted display that displays an image outputtedfrom the image generation device; and an input device that inputs anoperation indicative of the obstacle's entry into and exit from the stayarea, and transmits information regarding the obstacle's entry into andexit from the stay area to the image generation device.
 12. Aninformation presentation method of an image generation device,comprising: constructing a virtual world to be displayed on ahead-mounted display; generating, based on a position and a posture of ahead of a player wearing the head-mounted display, a display imagerepresenting the virtual world in a field of view corresponding to apoint of view of the player, and causing the head-mounted display todisplay the generated display image; acquiring information regardingpresence of an obstacle which is not related to the player in a spacewhere the player is able to move; and displaying an object in thevirtual world corresponding indicating the presence of the obstaclewhile the obstacle is present within the area.
 13. A non-transitorycomputer readable storage medium having stored thereon a program forcausing an image generation device execute processing comprising: aspace construction section that constructs a virtual world to bedisplayed on a head-mounted display; an image generation section thatgenerates, based on a position and a posture of a head of a playerwearing the head-mounted display, a display image representing thevirtual world in a field of view corresponding to a point of view of theplayer, and causes the head-mounted display to display the generateddisplay image; and an obstacle information acquisition section thatacquires information regarding presence of an obstacle which is notrelated to the player in a space where the player is located, wherein,while the obstacle is present within the space, the space constructionsection displays a first object in the virtual world corresponding tothe obstacle.